Skip to content
This repository was archived by the owner on Jul 4, 2023. It is now read-only.

Add comparison function #14

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
Draft

Add comparison function #14

wants to merge 9 commits into from

Conversation

ericphanson
Copy link
Owner

Based on #13.

This is a draft because it needs more work, but I figured I'd put up what I have. Given two BenchmarkHistogram objects (the result of @benchmark), it can plot the histograms one after the other using the same bins:

julia> using BenchmarkHistograms

julia> b1 = @benchmark sin(x) setup=(x = rand())
samples: 10000; evals/sample: 1000; memory estimate: 0 bytes; allocs estimate: 0
ns

 (7.12  - 7.34 ]  ██████████████████████████████▏6979
 (7.34  - 7.56 ]  ██▌581
 (7.56  - 7.78 ]  ▋124
 (7.78  - 7.99 ]  ▎57
 (7.99  - 8.21 ]  ▏6
 (8.21  - 8.43 ]  ▏2
 (8.43  - 8.65 ]  ▏1
 (8.65  - 8.86 ]  ▏4
 (8.86  - 9.08 ]  ▏9
 (9.08  - 9.3  ]  ▏11
 (9.3   - 9.52 ]  ████████▌1969
 (9.52  - 9.73 ]  ▍84
 (9.73  - 9.95 ]  ▍75
 (9.95  - 10.17]  ▎32
 (10.17 - 20.58]  ▍66

                  Counts

min: 7.125 ns (0.00% GC); mean: 7.724 ns (0.00% GC); median: 7.209 ns (0.00% GC); max: 20.583 ns (0.00% GC).

julia> b2 = @benchmark sin(x) setup=(x = 10rand())
samples: 10000; evals/sample: 999; memory estimate: 0 bytes; allocs estimate: 0
ns

 (7.76  - 8.22 ]  ████▎765
 (8.22  - 8.68 ]  ▏4
 (8.68  - 9.14 ]  ▏1
 (9.14  - 9.6  ]  ████████▎1521
 (9.6   - 10.07]  ██████████████████████████████▏5534
 (10.07 - 10.53]  ███████████▏2050
 (10.53 - 10.99]  ▎38
 (10.99 - 11.45]  ▏12
 (11.45 - 11.91]  ▏3
 (11.91 - 12.38]  ▏2
 (12.38 - 12.84]  ▏2
 (12.84 - 13.3 ]  ▏5
 (13.3  - 13.76]  ▏20
 (13.76 - 14.22]  ▎29
 (14.22 - 16.31]  ▏14

                  Counts

min: 7.758 ns (0.00% GC); mean: 9.796 ns (0.00% GC); median: 10.010 ns (0.00% GC); max: 16.308 ns (0.00% GC).

julia> comparison(b1, b2)
 (7.12  - 7.62 ]  ██████████████████████████████7578
 (7.62  - 8.12 ]  ▊167
 (8.12  - 8.62 ]  ▏4
 (8.62  - 9.12 ]  ▏14
 (9.12  - 9.62 ]  ████████1992
 (9.62  - 10.11]  ▊171
 (10.11 - 10.61]  ▏13
 (10.61 - 11.11]  ▏2
 (11.11 - 11.61]  ▏1
 (11.61 - 12.11]   0
 (12.11 - 12.6 ]   0
 (12.6  - 13.1 ]   0
 (13.1  - 13.6 ]   0
 (13.6  - 14.1 ]   0
 (14.1  - 20.58]  ▎58

 (7.12  - 7.62 ]   0
 (7.62  - 8.12 ]  ████767
 (8.12  - 8.62 ]  ▏6
 (8.62  - 9.12 ]  ▏1
 (9.12  - 9.62 ]  ███████▉1521
 (9.62  - 10.11]  ██████████████████████████████ 5837
 (10.11 - 10.61]  █████████▏1753
 (10.61 - 11.11]  ▎35
 (11.11 - 11.61]  ▏11
 (11.61 - 12.11]  ▏3
 (12.11 - 12.6 ]  ▏1
 (12.6  - 13.1 ]  ▏2
 (13.1  - 13.6 ]  ▏16
 (13.6  - 14.1 ]  ▎30
 (14.1  - 20.58]  ▏17

                  Counts

I think it should at least print a comparison of the summary stats as well, but also perhaps should plot them next to each other (reusing the bin labels) rather than consecutively. Of course, consecutive is a lot easier here since it's just more println's.

@codecov-commenter
Copy link

codecov-commenter commented May 23, 2021 •
edited
Loading

Codecov Report

Merging #14 (075e407) into eph/outliers (9d70bbf) will decrease coverage by 13.51%.
The diff coverage is 53.12%.

Impacted file tree graph

@@                Coverage Diff                @@
##           eph/outliers      #14       +/-   ##
=================================================
- Coverage        100.00%   86.48%   -13.52%     
=================================================
  Files                 3        4        +1     
  Lines                84      111       +27     
=================================================
+ Hits                 84       96       +12     
- Misses                0       15       +15
Impacted Files Coverage Δ
src/BenchmarkHistograms.jl 100.00% <ø> (ø)
src/comparison.jl 0.00% <0.00%> (ø)
src/simple_unicode_histogram.jl 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9d70bbf...075e407. Read the comment docs.

Base automatically changed from eph/outliers to main May 25, 2021 10:55
@kalmarek
Copy link

I'd expect the comparison (maybe better compare??) to do left-right alignment, e.g. (sorry, hand modified, no code to back this idea up:)

julia> comparison(b1, b2)
                                          0 (7.12  - 7.62 ]  ██████████████████████████████7578
                                    767████ (7.62  - 8.12 ]  ▊167
                                         6 ▏(8.12  - 8.62 ]  ▏4
                                         1 ▏(8.62  - 9.12 ]  ▏14
                              1521 ███████▉ (9.12  - 9.62 ]  ████████1992
        5837 ██████████████████████████████ (9.62  - 10.11]  ▊171
                             1753 █████████▏(10.11 - 10.61]  ▏13
                                        35 ▎(10.61 - 11.11]  ▏2
                                        11 ▏(11.11 - 11.61]  ▏1
                                         3 ▏(11.61 - 12.11]   0
                                         1 ▏(12.11 - 12.6 ]   0
                                         2 ▏(12.6  - 13.1 ]   0
                                        16 ▏(13.1  - 13.6 ]   0
                                        30 ▎(13.6  - 14.1 ]   0
                                        17 ▏(14.1  - 20.58]  ▎58

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ericphanson @codecov-commenter @kalmarek